The technology disclosed herein relates to data processing systems, and methods of operating such systems, in particular in relation to the display of sequences of frames (images) on a display of a data processing system.
FIG. 1 shows an exemplary system on chip (SoC) data processing system 8 that comprises a host processor comprising a central processing unit (CPU) 1, a graphics processing unit (GPU) 2, a display processor 3, and a memory controller 5. The exemplary data processing system may also comprise a video engine (not shown in FIG. 1). As shown in FIG. 1, these units communicate via an interconnect 4 and have access to off-chip memory 6. In this system, the graphics processing unit (GPU) 2 will render frames (images) to be displayed, and the display processor 3 will then provide the frames to a display panel 7 for display.
In use of this system, an application such as a game, executing on the host processor (CPU) 1 will, for example, require the display of frames on the display panel 7. To do this, the application will submit appropriate commands and data to a driver for the graphics processing unit (GPU) 2 that is executing on the CPU 1. The driver will then generate appropriate commands and data to cause the graphics processing unit (GPU) 2 to render appropriate frames for display and to store those frames in appropriate frame buffers, e.g. in the main memory 5. The display processor 3 will then read those frames into a buffer for the display from where they are then read out and displayed on the display panel 7 of the display.
The data processing system 8 will be configured to provide frames for display, and the graphics processing unit (GPU) 2 will correspondingly be configured to render frames, at an appropriate rate, such as 30 frames per second. Modern electronic devices increasingly use higher resolution displays. For example, displays with a “4k” resolution are becoming more common (a so-called “4k display”). However, providing sequences of frames of the required resolution for display by such devices, at the refresh rate of the display, presents some challenges.
Various rendering techniques have been proposed to allow frames of a resolution and rate required for output, i.e. display, to be produced even where the frame producer, e.g. graphics processor, is unable to produce fully rendered frames at such a resolution and rate. Such techniques typically involve the frame producer generating new data for only a portion of a frame, with some form of processing then being performed to derive additional data to fill in the “missing” data in the remainder of the frame, and provide an output frame of the required level of resolution.
One such technique is known as “upscaling”. In this technique, a lower resolution version of a frame is produced i.e. at a resolution level that the frame producer e.g. graphics processor is able to produce at the required rate. Thus, new data is generated in relation to only a portion of the frame. This lower resolution frame is then “upscaled” to provide a frame of a required resolution for output, (e.g. display). The process of upscaling may involve interpolating data in respect of portions, e.g. pixels, for which data has not been generated in the lower resolution version of the frame using the data in respect of adjacent portions e.g. pixels in the same frame.
Another technique which has been proposed more recently is known as “checkerboard rendering”. In one example of this technique, for a first frame, new data is generated in respect to one half of the frame, i.e. the black blocks of a checkerboard pattern. In the next (second) frame, new data is generated in respect of the other half of the frame i.e. the white blocks of the checkerboard pattern. In a simple implementation, the “missing” data for any particular frame is based on the existing data generated in relation to the corresponding blocks of the previous frame. Thus, in the example, the data for the black blocks, which is newly generated in respect of the first frame, is used in providing the data for the black blocks in the second frame, along with the newly generated data for the white blocks. In the next frame, the data for the white blocks will be data based on that generated for the previous frame, while the data for the black blocks will be newly generated in respect of the new frame, and so on.
When implementing a checkerboard technique, the “missing” data in respect of certain blocks in a particular frame is based on data (newly) generated in a preceding frame in respect of those blocks. The data may simply correspond to the previously generated data for the blocks, or may be determined using some form of interpolation based on the previously (newly) generated data. In some more recent developments in relation to checkerboard rendering, the “missing” data for certain blocks in a frame is determined by interpolation based on tracking the movement of objects e.g. primitives between frames. It is possible to record the identity of individual objects e.g. primitives within a frame. For example, such functionality is provided by some modern Application Programming Interfaces (APIs), such as Vulkan. The ability to record the identity of individual objects within a frame correspondingly allows those objects to be tracked as they move between frames. Motion vectors may be derived for the objects.
It has been found that using checkerboard rendering in which data (newly) generated in respect of different subsets of blocks in different frames is combined, e.g. using interpolation based on the motion of objects between frames, to produce a frame of a desired resolution for output, may provide an improved version of a frame of a desired resolution level for output than may be achieved simply using an upscaling process based on a lower resolution version of the frame. The checkerboard rendering may thus provide the ability to reduce load on a graphics processor when providing higher resolution output frames, but with better image quality than may be achieved using an upscaling process. It will be appreciated that in an upscaling process, “missing” data is determined based upon newly generated data in the same frame, while in a checkerboard rendering process the “missing” data is determined using data (newly) generated in relation to a preceding frame.
Nonetheless, the Applicant has recognised that there remains scope for improvement in techniques for rendering a sequence of frames to produce a sequence of frames for display.
Like reference numerals are used for like components where appropriate in the drawings.